In mobile networks, the movement of users and changes in user bandwidth requirements creates significant uncertainty as to the correct allocation of resources in a wireless access router, and on the links to and from the core network.
In traditional cellular networks this is solved by using MAC layer signaling to provide control loops for resource management both to and from the base station via the base station controller, and on through to the first IP QoS-aware device known as the Packet Data Switched Network (PDSN) in (3rd Generation Partnership Project 2 (3GPP2) terminology. The further the control loops extend up the network hierarchy, the longer the loops and feedback delay, resulting in less accurate control with the side-effect that either too much or too little traffic in each class may be delivered over the networks in each direction.
In a wireless network, the base station may be an IP-aware device, such as a wireless access router. In many cases, the base station is able to deliver IP Quality of Service (QoS) in both directions, both over the air to the mobile node (MN), e.g., mobile device, and over the backhaul link to the Operator Edge Router (OER). The OER is then typically connected back into the regional Point of Presence (POP) of the wireless operator and Internet Service Provider (ISP).
In any such IP-aware device, located at the edge of a dynamic network, and connected over costly access links, it is desirable to dimension the access link and wireless links at the wireless access router itself so that the wireless access router has the right amount of user traffic, of each traffic type, at any point in time, so that the air link and access link can be fully utilized and the buffering delay and packet drops in the wireless access router can be effectively managed. In effect, the buffering and queuing discipline in the wireless access router should have just enough of each class of packet for/from each user to meet the user's needs, and to compensate for the delay due to the variations in the radio channel over time, and to compensate for variations in the delay of packet delivery over the access link.
With the support of a multitude of traffic classes, such as voice, best effort data, interactive data, low loss data, etc., it is important to divide the access link bandwidth into partitions for each class. One commonly used technique for doing this is the IETF differentiated services model, as well as general hierarchical class-based schedulers such as CBQ, HFSC and Cisco CB-WFQ and CAR. Ideally, this partition should accurately reflect the actual mix of traffic flowing to avoid the situation where one user in a low priority partition ends up getting better service than a full complement of users in a higher priority partition.
It is therefore clear that it is desirable, to maximize utility and revenue, to dynamically adjust the partitions, whether class-based or diff-serv based, or based on any other partitioning technique, according to the real-time behavior of the users on each wireless access router in each direction of data transmission to/from the wireless devices.
In a mobile network based on mobile IP, incoming traffic to the home address of the MN goes via the MIP home agent where it is encapsulated and forwarded to the Care of Address of either the MN, or the MNs point of attachment known as a foreign agent (FA) or attendant. Meanwhile, outgoing traffic from a wireless access router usually goes un-encapsulated according to normal routing based on the IP destination address. In MIP reverse tunneling, the packets are instead first sent in a tunnel to the MIP Home Agent (HA). Therefore in the mobile network any control loop should be able to operate on encapsulated packets to and from the Home Agent.
In mobile IP, two foreign agents (FAs) also tunnel traffic between each other during hand-off as a result of MN movement between them. The source and destination addresses of the two tunnels are the pair of foreign agents involved in the hand-off.
These tunnels are used to forward data to the MN via the new FA, from the previous FA, and in addition, either a tunnel or a native IP route needs to be used between them to transfer MN state in a process known as Context Transfer. This hand-off traffic is again very dynamic and generally requires low loss and low latency in the network. In a handoff, the previous FA may act like a temporary HA in that it encapsulates packets to the CoA of the MN.
In a mobile network connected to other networks, or to other systems which use different resource management techniques, there will normally be a number of interface boxes or gateways (PSTN gateways, webfarm firewalls, corporate access routers, MPLS edge devices) that act as major traffic sources and sinks for the wireless access routers. In such systems it would be beneficial to extend the control loops out to these major elements so they can affect PSTN connection admission control, resource allocation, and scheduling on these elements across the range of class-based partitions.
In summary, there is a need to be able to load or ‘impedance match’ the resource segments in a wireless network in terms of per-class partitioned resources, buffering and scheduler activity, in both directions, so that the communication system can dynamically take account of the significant changes in resource requirements required by each wireless access router, based on its active users. This matching should use appropriate control loops and aggregation so that ideally the longer the distance from the wireless access router, the slower the control loop. Matching the control loop distance to rate of adjusted, e.g., control loop speed, can be important for control loop stability.
The prior art for an exemplary resource signaling system known as RSVP will now be described, without loss of generality of the invention.
RSVP is typically used end to end between two nodes to book resources between those nodes in either or both directions. A Path message is sent by one node to discover the routing of packets from that node to the receiver, and to identify the characteristics of the resources on that routing, or Path. The Path message installs path state as it traverses the nodes that it visits that are running RSVP software. The receiver then replies with a reservation (RESV) message which follows the path state in the reverse direction and books the resources in those nodes for packets sent in the same direction as the Path message (i.e. in the opposite direction to the RESV message). If those resources do not exist then a RESV error is returned to the receiver that sent the RESV message.
Existing RSVP is composed of two uni-directional processes for the upstream and downstream directions. Prior art from Cable Labs for enhancing standard RSVP signaling, in a cable modem network, includes the following bi-directional modifications to RSVP over a single hop. The Cable Labs work can be applied over any link between a node and an Access Router. In a wireless based system, MN issues a Path message to the AR as well as a premature RESV for the downstream resources. The AR then issues a premature RESV for the upstream resources in response to the Path from the MN and may optionally, in addition, issue a premature Path in response to the RESV from the MN, so that the MN can forward this Path on to any nodes beyond the MN. The RESV from the MN will also cause the Correspondent Node (CN) of the MN to reply with the Path message that would normally have started the uplink resource reservation process. This Path message might subsequently cause a change in the RESV from the MN. When the Path from the MN reaches the actual CN destination, then the true RESV message will be issued back to the MN, which will modify the premature RESV sent from the AR to the MN. In this way, the reservation of resources on the link between the MN and the AR can be speeded up using bi-directional RSVP. Note that in the Cable Labs work, RSVP tunneling or RSVP Aggregation based on the Differentiated Services Code Point (DSCP), discussed next, is not considered or suggested.
FIGS. 1 and 2 illustrate two common aggregation techniques based on RSVP, known as RSVP tunneling and RSVP Aggregation. As is well known in the art, an aggregation peer exists at either end of an aggregation region, through which multiple parallel aggregates are defined between those peers. An aggregate is defined both by some definition of the packets that are members of that aggregate, called the constituent flows, as well as some proportion of the link resources shared by that aggregate, defined by some function F. The combination of the resource functions in a specific direction over the link at any point in time must be equal to the total resources associated with that link, where resources comprise both bandwidth and latency. Hence one aggregate can be assigned some portion of bandwidth and some latency bound that will be useful for VoIP traffic, whilst another aggregate can get the remaining resources and a variable latency bound for best effort traffic. The mapping between the aggregates and the constituent flows in each aggregate on a link is handled by a constituent classifier, which is communicated to the ingress Aggregation Peer, for example, using constituent resource signaling. This classifier defines the constituent packet flow and the target aggregate for that flow. Each aggregate will then have a second aggregate classifier that defines for all routers between the aggregation peers the unique identifiers for each aggregate. A typical unique identify could be a specific Differentiated Services Code Point (DSCP) value or a specific tunnel encapsulation. This aggregate classifier is communicated through the aggregation region using aggregate resource signaling, which avoids the aggregation region having to know anything about the constituent flows in an aggregate and specifically all the individual constituent classifiers. This aggregate resource signaling also includes the amount of resources assigned to the aggregate identified by the aggregate classifier. Such aggregate and constituent classifiers are henceforth referred to as aggregation classifiers.
An admission control procedure is typically undertaken at the aggregation peers whereby constituent resource signals are mapped into a specific aggregate and if necessary the resources in that aggregate adjusted based on the resource requirements of the constituent flow. If this mapping is undertaken by the egress aggregation peer then the mapping needs to be communicated to the ingress peer so that constituent flows can be uniquely marked to identify them with the assigned aggregate. If the resources for the aggregate are booked by the egress peer back to the ingress peer then the egress peer needs to know both the constituent resource requirements and the target aggregate for that constituent. Each aggregate typically has a resource threshold indicating the maximum amount of resource assigned to that aggregate, a portion of which is being employed at any point in time by the constituent flows mapped into that aggregate. This ensures that constituent resource signals and associated admission control procedures do not always result in additional aggregate resource signaling to increase/decrease aggregate resources given the change in the constituent flows. This damps and decouples the aggregate resource signaling from the constituent resource signaling.
In FIG. 1, RSVP tunneling has at least one IP-IP tunnel 501 between the two aggregation peers through which an end to end path (e2e_Path) message 504 can flow. IP-IP tunnel 501 has a tunnel path 508. Any constituent flows that are part of the tunnel 501 aggregate are mapped into that tunnel 501 at the aggregate ingress node (Rentry) 502 and deaggregated at the egress (Rexit) 503. The mapping between the constituent flow and the aggregate is communicated in a session_association object in the constituent Path message (e2e_Path+Session_assoc_Obj) 507 which is forwarded to Rexit 503. The aggregation ingress node (Rentry) 502 has a constituent classifier for each constituent flow along with the target resources for that flow and the target aggregate for that flow. The tunnel aggregate then has an aggregation classifier, which uniquely identifies the tunnel encapsulation, the constituent flows (packets) that are part of that tunnel aggregate, as well as the resources for that tunnel 501. The aggregate classifier, tunnel encapsulation, and resources are communicated to all the routers between the two aggregation peers. Any end to end resource signaling associated with a specific constituent flow can be used to increase or decrease the resources for each constituent flow, and the resources for the tunnel aggregate containing that constituent flow. The end to end (level 0) resource signaling for the constituent flow is then hidden from the intermediate routers (Rint) 506 so that they do not additionally and incorrectly modify the resources in the routers that form the tunnel 501 such as in the intermediate router (Rint) 506. This is naturally achieved by the tunnel header. Multiple parallel tunnels can be used between the same pair of aggregation peers to group constituent flows into a smaller number of aggregates that share similar resource characteristics so as to obtain statistical multiplexing gains and to reduce the resource processing and signaling load in the intermediate routers 506. The end to end (e2e) reservation (e2eResv) signal 505 is not normally forwarded into the tunnel 501 by Rexit 503 until the amount of resources additionally required by the tunnel for that end to end message 504 has been reserved by the Tunnel Resv message (Tunnel Resv+Resv_Conf_Obj) 509 and confirmed by the Resv Confirm message (Tunnel Resv_Confirm) 510 in all routers within the tunnel 501 serving the matching aggregate. If insufficient resources are present in the tunnel 501, indicated by Tunnel ResvErr message 511, then the Rexit node 503 alternatively triggers the e2eResvErr message 512 to the end to end source of the e2e Resv message. The tunnel header for each of the parallel aggregates can be differentiated in the intermediate routers 506, to enable the correct aggregate resources to be employed, by use of either a unique User Datagram Protocol (UDP) header, a unique (IP security Protocol (IPSEC) Security Parameter Index (SPI) or a unique Differentiated Services Code Point (DSCP) in the outer header of the tunnel 501.
In FIG. 2, RSVP Aggregation is shown. This aggregation technique does not necessarily use a tunnel 501, but can simply remark the DSCP of arriving packets to show which DSCP aggregate they are a member of. The absence of a tunnel 501 between the aggregation peers means that to hide the end to end (level 0) resource signaling, that is associated with a constituent flow, from the intermediate routers (Rint) 506, the resource signaling itself must carry a parameter that informs the intermediate routers 506 that they should ignore the resource signal. This ‘parameter’ must be enabled at the ingress aggregation node (RAgg) 601 and disabled at the egress deaggerator node (RdAgg) 603 so that only intermediate nodes 506 are affected, and downstream nodes beyond the aggregate egress 603 can still examine the end to end resource signal. The absence of a tunnel 501 also requires the level of the aggregate to be known at both ends by other means. In RSVP, the ‘parameter’ is effected by changing the protocol Id, marking the router alert option with ‘RSVP_end_to_end_ignore’, shown in e2e_Path+e2E_ignore Message 604, and including in the router alert option the aggregation level, so that only the deaggregating router 603 for that aggregate level demotes the resource signaling packet to the next lowest aggregate level. When the last deaggregator is reached, in a hierarchy of aggregation and deaggregation nodes, then the protocol Id will be converted back to normal RSVP and ‘end_to_end_ignore’ removed as shown in e2e_Path message 504. The ingress aggregation node (Ragg) 601 has resources and a constituent classifier for each constituent flow, and an aggregation classifier and resources for the DSCP aggregate that identifies the required DSCP value for that aggregate. This information is carried by the Agg Path (DSCP) message 606 and the Agg Resv message 607. If the aggregate includes a tunnel header then the tunnel encapsulation is also described by the aggregation classifier. The aggregate classifier mapping for each constituent flow is however in this case determined by the Egress Deaggregator node (RdAgg) 603 by communicating the mapping back to the Aggregator 601 in a constituent PathErr message 605, in the case of a new aggregate, or an Aggregate Resv message 607 in the case of a previously established aggregate, along with the aggregate identifier which is typically a DSCP value carried in an RSVP DCLASS object [DCLASS]. This is shown in FIG. 2 as e2e Path+New_Agg+DSCP_obj message 605 or Agg_Resv (DSCP)+Resv_Conf_obj Message 607 respectively, where the Resv_Conf object requests a confirmation of the availability of the additional aggregate resources back to the RdAgg. Once again the e2e_Resv message 505 is not forwarded back towards the RAgg 601 until the matching resources have been booked in all routers within the aggregate such as Rint 506, indicated by the receipt of the Resv_Confirm message 610. Error messages 611 and 512 are alternatively used to indicate resource reservation failure.
Existing RSVP tunneling and RSVP Aggregation support functionality to automatically discover the aggregation peers, to communicate the mappings between the aggregate and constituent flow, to communicate the resources for each aggregate, to aggregate constituent RSVP signaling into the RSVP aggregate signaling, and to then hide that signaling from the intermediate RSVP router nodes. The mappings are controlled by the aggregate ingress node (Rentry) 502 in RSVP tunneling whilst it is controlled by the egress deaggregator (RdAgg) node 603 in RSVP DSCP aggregation. Both techniques can use policy management techniques to configure resources and mappings into the aggregation peers for constituent flows that do not have associated resource signals. In addition, both techniques can assign what can be termed threshold resources to aggregates in advance of constituent resource signals so that aggregate resource signals are not triggered by each constituent flow resource signal event. This decoupling provides for more efficient signaling and a more stable system. Finally, both systems use the constituent flow resource signals to communicate the mapping of that constituent flow to a specific aggregate.
While the known aggregation techniques provide for the decoupling of the allocation of resources to aggregates and the assignment of flows to aggregates, the way in which these operations is done it not well suited for the dynamic nature of a mobile environment. In the known systems, aggregates flow between end nodes. Each end node controls, e.g., determines, resource allocation and assignment of flows to aggregates in a single direction. Thus, in the case where there are flows between a pair of aggregate end nodes in two different directions, e.g., bi-directional flows with packets being communicated in both an upstream and a downstream direction, different nodes end up being in control of which flows are assigned to particular upstream and downstream aggregates. In addition, in known systems one aggregate end node is responsible for determining the distribution of upstream resources among upstream aggregates and the other end node is responsible for determining the distribution of downstream resources among downstream aggregates.
The known control technique of making end nodes each responsible for controlling flow aggregation and resource allocation to aggregate packet flows in a single direction but not both, is based on the idea that different ends will be better suited to make such controlling decisions since the flows are in different directions and different ends should, presumably, be better aware of the factors which should be used in making such decisions for a given direction.
While the known aggregation techniques work adequately for many existing networks where fixed, relatively stable communications connections are used to couple end nodes to the network, they are not as well suited as may be desired for mobile networking. In the case of mobile systems, the access link between an access router and a core node represents a point which is likely to be subject to congestion due to limited resources for packet flows. This is because, generally, links further in the core have greater bandwidth capability than links further out in the network.
Demands for resources and the types of packet flows passing over the link between an access node and a core node can vary frequently based on a host of wireless link conditions and/or changes in the number and/or type of mobile terminals being serviced by the access node at any given time. This is because, for example, multiple end nodes, e.g., mobile terminals, may enter and leave a cell in a relatively short amount of time. Changes in signal interference and/or the type of data being communicated to/from mobiles can have a significant impact on the type and amount of packet traffic to be communicated over the link coupling the access node to a core node. In mobile systems, scheduling decisions in regard to scheduling uplink and/or downlink transmissions over wireless terminals can have a significant impact on not only the demand for resources on the access link coupling an access node to a core node but also on the mix of packet types and/or data packet priority levels which will be found in the packet traffic to be communicated over the end node.
Unfortunately, given communications delays that may exist between a core node and the access node to which it is connected and/or bandwidth constraints, it may not be practical to communicate all the information available at the access node in a time frame that would be particularly useful to the core node to make packet flow to aggregate assignments and/or resource to aggregate assignment decisions at the core node. Thus, if the known aggregation techniques are applied, the core node will make aggregation and resource to aggregate assignments relating to flows in one direction using information that is not the most current and/or does not reflect a fully informed decision since all the information available at the access node will normally not be available at the core node for decision making purposes.
As discussed above, in systems using MIPv4 or MIPv6, tunnels are often used to facilitate the redirection of packets to a mobile node which is attached to the network via an access router which is located outside its home domain. In such systems a tunnel for communicating IP packets to a mobile is normally established between the mobile's Home Agent (HA) and either the access router to which the mobile node is attached or the mobile node itself. Given that tunnels to a mobile node's HA extend through the core of the network, they will pass over the access link which couples the access router to which a mobile node is attached. Thus, flows of packets corresponding to a tunnel from a mobile node's HA to either the access router used to connect the mobile node to the network or to mobile node itself will be one of the flows passing over the link between the access router and core node, e.g., the first node in the core to which the access router is connected.
Management signals used to manage resource and/or allocation of packet flows to tunnels extending to a home agent may be useful in managing resources on links over which the tunnel passes. Unfortunately, the tunneling process normally results in such tunnel management signals being hidden from intermediate nodes, e.g., as a result of encapsulation. From a management standpoint, it would be desirable if tunnel management information could be made available to intermediate nodes, e.g., core nodes, which could then use the signals for flow management purposes.
In view of the above discussion, it should be apparent that there is a need for improved methods of controlling access link resources and the flow of packets between an access node, e.g., access router which serves as the point of attachment for mobile nodes, and a corresponding network node, e.g., the first core node to which a wireless access router is connected.
It is desirable that at least some new methods and/or apparatus facilitate using air link information, e.g., air link utilization information, scheduling information, current air link bandwidth, the number of mobile nodes in a cell which may use air link resources, etc., to control and/or influence the allocation of access link resources between the access node and core node. To reduce and/or eliminate control signaling requirements and/or delays it would be beneficial if, in at least some embodiments, control of access link resources be capable of being controlled primarily or at least initially from the access router for packet flows in both directions to/from the access node over the link to a core node for which resources are being allocated. In addition there is a need for improved messages and/or ways of communicating information relevant to aggregation of packet flows and/or allocation of resources over links where congestion may occur. There is also a need for, at least in some cases, methods and/or apparatus which can use information and/or control signals relating to tunnels to a mobile node's home agent when making decisions about allocating resources between links over which the tunnel or tunnels to the mobile node's home agent are routed.